Burn-in content signaling mechanism for avoiding text overlap

ABSTRACT

An encoder for use with video data, first supplemental data, second supplemental data, and a display device. The encoder includes: a memory; and a processor configured to execute instructions stored on the memory to cause the encoder to: generate modified video data based on an incorporation of the first supplemental data into the video data such that the supplemental video will be displayed at a predetermined location when the modified video is displayed on the display device; generate metadata associated with the predetermined location; generate an encoded transport stream including the modified video data, the metadata and the second supplemental data; and transmit the encoded transport stream to the display device so as to enable the display device to decode the encoded transport stream and display the modified video and the second supplemental video such that the second supplemental video is not displayed at the predetermined location.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Provisional App. No.63/285,862 filed Dec. 3, 2021, the content of which is incorporatedherein by reference in its entirety.

BACKGROUND

Embodiments of the invention relate to encoding and decoding data.

SUMMARY

Aspects of the present invention are drawn to an encoder for use withvideo data, first supplemental data, second supplemental data, and adisplay device, the video data corresponding to a video to be displayedon the display device, the first supplemental data corresponding to asupplemental video, the first supplemental data to be incorporated intothe video data so as to generate modified video data corresponding to amodified video to be displayed on the display device, the secondsupplemental data corresponding to a second supplemental video to besuperimposed on the modified video to be displayed on the displaydevice, the encoder including: a memory; a processor configured toexecute instructions stored on the memory to cause the encoder to:generate the modified video data based on an incorporation of the firstsupplemental data into the video data such that the supplemental videowill be displayed at a predetermined location when the modified video isdisplayed on the display device; generate metadata associated with thepredetermined location; generate an encoded transport stream includingthe modified video data, the metadata and the second supplemental data;and transmit the encoded transport stream to the display device so as toenable the display device to decode the encoded transport stream anddisplay the modified video and the second supplemental video such thatthe second supplemental video is not displayed at the predeterminedlocation.

In some embodiments, the processor is configured to execute instructionsstored on the memory to cause the encoder to transmit the encodedtransport stream to the display device so as to enable the displaydevice to decode the encoded transport stream and display the modifiedvideo and the second supplemental video such that the secondsupplemental video is displayed at a second predetermined location,wherein the predetermined location is different from the secondpredetermined location.

In some embodiments, the processor is configured to execute instructionsstored on the memory to cause the encoder to transmit the encodedtransport stream to the display device so as to enable the displaydevice to decode the encoded transport stream and display the modifiedvideo and the second supplemental video such that the secondsupplemental video is displayed at a second location, wherein thepredetermined location is different from the second location.

In some embodiments, the processor is configured to execute instructionsstored on the memory to additionally cause the encoder to generate theencoded transport stream including the audio data.

Other aspects of the present disclosure are drawn to a method of usingan encoder with video data, first supplemental data, second supplementaldata, and a display device, the video data corresponding to a video tobe displayed on the display device, the first supplemental datacorresponding to a supplemental video, the first supplemental data to beincorporated into the video data so as to generate modified video datacorresponding to a modified video to be displayed on the display device,the second supplemental data corresponding to a second supplementalvideo to be superimposed on the modified video to be displayed on thedisplay device, the method including: generating, via a processorconfigured to execute instructions stored on a memory, the modifiedvideo data based on an incorporation of the first supplemental data intothe video data such that the supplemental video will be displayed at apredetermined location when the modified video is displayed on thedisplay device; generating, via the processor, metadata associated withthe predetermined location; generating, via the processor, an encodedtransport stream including the modified video data, the metadata and thesecond supplemental data; and transmitting, via the processor, theencoded transport stream to the display device so as to enable thedisplay device to decode the encoded transport stream and display themodified video and the second supplemental video such that the secondsupplemental video is not displayed at the predetermined location.

In some embodiments, the method, wherein the transmitting the encodedtransport stream to the display device includes transmitting the encodedtransport stream to the display device so as to enable the displaydevice to decode the encoded transport stream and display the modifiedvideo and the second supplemental video such that the secondsupplemental video is displayed at a second predetermined location, andwherein the predetermined location is different from the secondpredetermined location.

In some embodiments, the method, wherein the transmitting the encodedtransport stream to the display device includes transmitting the encodedtransport stream to the display device so as to enable the displaydevice to decode the encoded transport stream and display the modifiedvideo and the second supplemental video such that the secondsupplemental video is displayed at a second predetermined location, andwherein the predetermined location is different from the secondlocation.

In some embodiments, the method, for further use with audio dataassociated with the first supplemental data, wherein the generating, viathe processor, the encoded transport stream including the modified videodata, the metadata and the second supplemental data includes generatingthe encoded transport stream including the audio data.

Other aspect of the present disclosure are drawn to a non-transitory,computer-readable media having computer-readable instructions storedthereon, the computer-readable instructions being capable of being readby an encoder for use with video data, first supplemental data, secondsupplemental data, and a display device, the video data corresponding toa video to be displayed on the display device, the first supplementaldata corresponding to a supplemental video, the first supplemental datato be incorporated into the video data so as to generate modified videodata corresponding to a modified video to be displayed on the displaydevice, the second supplemental data corresponding to a secondsupplemental video to be superimposed on the modified video to bedisplayed on the display device, wherein the computer-readableinstructions are capable of instructing the encoder to perform themethod including: generating, via a processor configured to executeinstructions stored on a memory, the modified video data based on anincorporation of the first supplemental data into the video data suchthat the supplemental video will be displayed at a predeterminedlocation when the modified video is displayed on the display device;generating, via the processor, metadata associated with thepredetermined location; generating, via the processor, an encodedtransport stream including the modified video data, the metadata and thesecond supplemental data; and transmitting, via the processor, theencoded transport stream to the display device so as to enable thedisplay device to decode the encoded transport stream and display themodified video and the second supplemental video such that the secondsupplemental video is not displayed at the predetermined location.

In some embodiments, the non-transitory, computer-readable media,wherein the computer-readable instructions are capable of instructingthe encoder to perform the method wherein the transmitting the encodedtransport stream to the display device includes transmitting the encodedtransport stream to the display device so as to enable the displaydevice to decode the encoded transport stream and display the modifiedvideo and the second supplemental video such that the secondsupplemental video is displayed at a second predetermined location, andwherein the predetermined location is different from the secondpredetermined location.

In some embodiments, the non-transitory, computer-readable media,wherein the computer-readable instructions are capable of instructingthe encoder to perform the method wherein the transmitting the encodedtransport stream to the display device includes transmitting the encodedtransport stream to the display device so as to enable the displaydevice to decode the encoded transport stream and display the modifiedvideo and the second supplemental video such that the secondsupplemental video is displayed at a second predetermined location, andwherein the predetermined location is different from the secondlocation.

In some embodiments, the non-transitory, computer-readable media, forfurther use with audio data associated with the first supplemental data,wherein the computer-readable instructions are capable of instructingthe encoder to perform the method wherein the generating, via theprocessor, the encoded transport stream including the modified videodata, the metadata and the second supplemental data includes generatingthe encoded transport stream including the audio data.

Other aspects of the present disclosure are drawn to a decoder for usewith an encoded transport stream and a display device, the encodedtransport stream including modified video data, metadata, and secondsupplemental data, the modified video data corresponding to a modifiedvideo to be displayed on the display device and being based on anincorporation of first supplemental data into video data, the firstsupplemental data corresponding to a supplemental video to be displayedat a predetermined location when the modified video is displayed on thedisplay device, the video data corresponding to a video to be displayedon the display device, the second supplemental data corresponding to asecond supplemental video to be superimposed on the modified video to bedisplayed on the display device, the metadata being associated with thepredetermined location, the decoder including: a memory; a processorconfigured to execute instructions stored on the memory to cause thedecoder to: decode the encoded transport stream to generate the modifiedvideo data, the metadata, and the second supplemental data; and providethe modified video data and the second supplemental data to the displaydevice such that, based on the meta data, the second supplemental datais not displayed at the predetermined location.

In some embodiments, the processor is configured to execute instructionsstored on the memory to cause the decoder to transmit the encodedtransport stream to the display device so as to enable the displaydevice to provide the modified video data and the second supplementaldata to the display device such that, based on the meta data, the secondsupplemental data is displayed at a second predetermined location, andwherein the predetermined location is different from the secondpredetermined location.

In some embodiments, the processor is configured to execute instructionsstored on the memory to cause the decoder to transmit the encodedtransport stream to the display device so as to enable the displaydevice to provide the modified video data and the second supplementaldata to the display device such that, based on the meta data, the secondsupplemental data is displayed at a second location, and wherein thepredetermined location is different from the second location.

In some embodiments, the decoder, for further use with audio dataassociated with the first supplemental data, wherein the processor isconfigured to execute instructions stored on the memory to additionallycause the decoder to decode the encoded transport stream to generate theaudio data.

Other aspects of the present disclosure are drawn to a method of using adecoder with an encoded transport stream and a display device, theencoded transport stream including modified video data, metadata, andsecond supplemental data, the modified video data corresponding to amodified video to be displayed on the display device and being based onan incorporation of first supplemental data into video data, the firstsupplemental data corresponding to a supplemental video to be displayedat a predetermined location when the modified video is displayed on thedisplay device, the video data corresponding to a video to be displayedon the display device, the second supplemental data corresponding to asecond supplemental video to be superimposed on the modified video to bedisplayed on the display device, the metadata being associated with thepredetermined location, the method including: decoding, via a processorconfigured to execute instructions stored on a memory, the encodedtransport stream to generate the modified video data, the metadata, andthe second supplemental data; and providing, via the processor, themodified video data and the second supplemental data to the displaydevice such that, based on the meta data, the second supplemental datais not displayed at the predetermined location.

In some embodiments, the method, wherein the providing, via theprocessor, the modified video data and the second supplemental data tothe display device includes providing the modified video data and thesecond supplemental data to the display device such that, based on themeta data, the second supplemental data is displayed at a secondpredetermined location, and wherein the predetermined location isdifferent from the second predetermined location.

In some embodiments, the method, wherein the providing, via theprocessor, the modified video data and the second supplemental data tothe display device includes providing the modified video data and thesecond supplemental data to the display device such that, based on themeta data, the second supplemental data is displayed at a secondlocation, and wherein the predetermined location is different from thesecond location.

In some embodiments, the method, for further use with audio dataassociated with the first supplemental data, wherein the decoding theencoded transport additionally generates the audio data.

Other aspects of the present disclosure are drawn to a non-transitory,computer-readable media having computer-readable instructions storedthereon, the computer-readable instructions being capable of being readby a decoder for use with an encoded transport stream and a displaydevice, the encoded transport stream including modified video data,metadata, and second supplemental data, the modified video datacorresponding to a modified video to be displayed on the display deviceand being based on an incorporation of first supplemental data intovideo data, the first supplemental data corresponding to a supplementalvideo to be displayed at a predetermined location when the modifiedvideo is displayed on the display device, the video data correspondingto a video to be displayed on the display device, the secondsupplemental data corresponding to a second supplemental video to besuperimposed on the modified video to be displayed on the displaydevice, the metadata being associated with the predetermined location,wherein the computer-readable instructions are capable of instructingthe decoder to perform the method including: decoding, via a processorconfigured to execute instructions stored on a memory, the encodedtransport stream to generate the modified video data, the metadata, andthe second supplemental data; and providing, via the processor, themodified video data and the second supplemental data to the displaydevice such that, based on the meta data, the second supplemental datais not displayed at the predetermined location.

In some embodiments, the non-transitory, computer-readable media,wherein the computer-readable instructions are capable of instructingthe encoder to perform the method wherein the providing, via theprocessor, the modified video data and the second supplemental data tothe display device includes providing the modified video data and thesecond supplemental data to the display device such that, based on themeta data, the second supplemental data is displayed at a secondpredetermined location, and wherein the predetermined location isdifferent from the second predetermined location.

In some embodiments, the non-transitory, computer-readable media,wherein the computer-readable instructions are capable of instructingthe encoder to perform the method wherein the providing, via theprocessor, the modified video data and the second supplemental data tothe display device includes providing the modified video data and thesecond supplemental data to the display device such that, based on themeta data, the second supplemental data is displayed at a secondlocation, and wherein the predetermined location is different from thesecond location.

In some embodiments, the non-transitory, computer-readable media, forfurther use with audio data associated with the first supplemental data,wherein the computer-readable instructions are capable of instructingthe encoder to perform the method wherein the decoding the encodedtransport additionally generates the audio data.

BRIEF SUMMARY OF THE DRAWINGS

The accompanying drawings, which are incorporated in and form a part ofthe specification, illustrate example embodiments and, together with thedescription, serve to explain the principles of the invention. In thedrawings:

FIG. 1A illustrates a display device at a time t₀;

FIG. 1B illustrates display device at a time t₁;

FIG. 1C illustrates display device at a time t₂;

FIG. 2 illustrates an exploded view of a prior art encoder, a prior arttransport stream, and a prior art decoder;

FIG. 3 illustrates a method for encoding data and transporting it to adecoder in accordance with aspects of the present disclosure;

FIG. 4 illustrates an exploded view of an encoder, a transport stream,and a decoder in accordance with aspects of the present disclosure;

FIG. 5 illustrates a method for receiving encoded data from an encoderand decoding data in accordance with aspects of the present disclosure;

FIG. 6A illustrates a display device at a time t₀ in accordance withaspects of the present disclosure;

FIG. 6B illustrates a display device at a time t₁ in accordance withaspects of the present disclosure; and

FIG. 6C illustrates a display device at a time t₂ in accordance withaspects of the present disclosure.

FIGS. 7 and 8 each illustrates an exploded view of an encoder, atransport stream, and a decoder in accordance with alternate aspects ofthe present disclosure.

DETAILED DESCRIPTION

Video content may be provided to a video display device to be watched bya user. Some video content, such as movies, may include textualinformation like closed captions and subtitles that are displayed withinthe video to provide viewers with additional, interpretive information,or oftentimes simply a textual translation of the audio track. Thisinformation may be provided to a video receiver as text strings throughTeletext or Close captions in analog video signals, or through datapackets in Moving Picture Experts Group (MPEG) video streams.Alternatively, the text may be burned (embedded) within the video framesby an encoder prior to its transmission. This embedded text may bereferred to as burnt-in static text regions, which will be described ingreater detail with reference to FIGS. 1A-1B.

Specifically, FIG. 1A illustrates a display device 100 at a time t₀ thatdisplays an image 102, which represents one of a plurality of sequentialframes of a movie, generated from video data provided to the displaydevice. FIG. 1B, in turn, illustrates the display device 100 at a timet₁ while it displays an image 104 comprising another frame of the movie.On the bottom of the image 104 is a predetermined subtitle location 106,within which are subtitles 108. For purposes of discussion, let themovie that is being watched by the viewer have an associated audio trackthat is not in the language of the viewer. For example, for purposes ofdiscussion, let the movie have an audio track that is in French.Further, when this video was encoded, there was a decision by thedistribution company that the largest market speaks English and notFrench. Therefore, to please the largest English-speaking market,English subtitles corresponding to the French audio track are added tothe video. More specifically, the English subtitles are burned-into thevideo, wherein they cannot be removed. In particular, supplemental datacorresponding to supplemental video, which in this case is supplementalsubtitle image data corresponding to the subtitles, is incorporated inthe video data of the original video to create modified video data,which will generate a modified video—i.e., the original video with thesubtitles burned therein. Accordingly, as compared to image 102 of FIG.1A, image 104 of FIG. 1B includes the content of image 102 in additionto subtitles 108 at location 106 of image 102.

In some cases, a viewer may want a different set of subtitles. Forexample, and for purposes of discussion, consider the example discussedabove with reference to FIG. 1B, where the video has audio track that isin French, and English subtitles corresponding to the French audio trackare burned into the video and cannot be removed. Still further, supposethe viewer doesn't understand French or read English but, in thisexample, only understands and reads Spanish. Luckily, the video contentis provided with additional optional subtitle choices, where one of thesubtitle choices is Spanish. In such a case, the user may choose theSpanish subtitles. However, the newly-chosen Spanish subtitles are notburned into the video; rather, the supplemental data corresponding tothe Spanish subtitles is added to the modified video data such that theSpanish subtitles are superimposed on the modified video data.

However, there are many instances wherein the second supplemental data,which in this case is the data corresponding to the Spanish subtitles,is decoded so as to display the Spanish subtitles in an area thatoverlaps the original English burned-in subtitles. With the subtitlesoverlapping, the user will have trouble reading the subtitles that theyturned on. For example, FIG. 1C shows the display device 100 at a timet₂ when it displays a frame 104, at the bottom of which is predeterminedsubtitle location 106 and burned-in English subtitles 108. In addition,the user-selected Spanish subtitles 114 are shown within subtitlelocation 112, and overlap the burned-in subtitles 108. Accordingly, itis very hard for the viewer to read the Spanish subtitles because theyoverlap the English subtitles.

The encoding and decoding of video data as discussed above withreference to FIGS. 1A-C will now be described in greater detail withreference to FIG. 2 , which shows an exploded view of a prior artencoder 200, a prior art transport stream 218, and a prior art decoder230. Encoder 200 preferably includes a controller 201 and a memory 213,and receives video data 202, audio data 204, audio data 206, subtitledata 208, and subtitle data 210. In some cases, at least one ofcontroller 201 and memory 213 may be implemented as a computer havingtangible computer-readable media for carrying or havingcomputer-executable instructions or data structures stored thereon.Memory 213 additionally preferably contains instructions to allowcontroller 201 to encode data; and transport the encoded data to decoder230.

Controller 201 may be implemented as a hardware processor such as amicroprocessor, a multi-core processor, a single core processor, a fieldprogrammable gate array (FPGA), a microcontroller, an applicationspecific integrated circuit (ASIC), a digital signal processor (DSP), orother similar processing device capable of executing any type ofinstructions, algorithms, or software for controlling the operation andfunctions of encoder 200.

Video data 202 may be any form a visual media, a non-limiting example ofwhich being a movie. Audio data 204 and audio data 206 may be any formof auditory media that corresponds with video data 202. For purposes ofdiscussion only, let audio data 204 correspond to French audiocorresponding to video data 202, and let audio data 206 correspond toEnglish audio corresponding to video data 202. Subtitle data 208 andsubtitle data 210 correspond with subtitles that may be displayed. Forpurposes of discussion only, let subtitle data 208 correspond to Spanishsubtitles for either of French audio data 204 or English audio data 206,and let subtitle data 210 correspond to English subtitles for either ofFrench audio data 204 or English audio data 206. Video data 202, Frenchaudio data 204, English audio data 206, Spanish subtitle data 208, andEnglish subtitle data 210 may in some preferred embodiments be receivedby encoder 200 from an external source.

Video data 202 and English subtitle data 210 may be stored in memory213, where it is stored as video data 214. Video data 214 may containEnglish subtitle data 210, which is converted to burn-in subtitle data216. As it is burned in, burn-in subtitle data 216 alters video data202, thereby converting it to video data 214.

Encoder 200 is a content provider that encodes data for the purpose ofdistributing it. In some cases, the content will be delivered on aphysical media, a non-limiting example of which being a digitalversatile disc (DVD). In some cases, the content will be providedthrough the use of a cable modem termination system (CMTS), in which amedia provider provides content to the CMTS, which then provides it toencoder 200.

Encoder 200 encodes video data 214, which has stored therein burned-insubtitle data 216, French audio data 204, English audio data 206, andSpanish subtitle data 208, and transmits the encoded data to decoder 230as transport stream 218 through communication channel 217. Communicationchannel 217 may be any known type of wired or wireless communicationchannel. In a broad sense, communication channel 217 may incorporate anycombination of: wired communication channels, a non-limiting example ofwhich includes Data Over Cable Service Interface Specification (DOCSIS),wireless communication channels, a non-limiting example of whichincludes a cellular network, and a media communication channel, anon-limiting example of which includes a DVD.

As shown in FIG. 2 transport stream 218 includes encoded video data 220,which has stored therein encoded burn-in subtitle data 222, and alsoincludes encoded audio data 224, encoded audio data 226, and encodedsubtitle data 228. Transport stream 218 contains the encoded data sentfrom encoder 200 to decoder 230.

As shown in FIG. 2 , decoder 230 includes a controller 231 and a memory233. Memory 233 has stored therein video data 214, which includesburned-in subtitle data 216, French audio data 204, English audio data206, and Spanish subtitle data 208. In some cases, at least one ofcontroller 231 and memory 233 may be implemented as a computer havingtangible computer-readable media for carrying or havingcomputer-executable instructions or data structures stored thereon.Controller 231 may be implemented as a hardware processor such as amicroprocessor, a multi-core processor, a single core processor, a fieldprogrammable gate array (FPGA), a microcontroller, an applicationspecific integrated circuit (ASIC), a digital signal processor (DSP), orother similar processing device capable of executing any type ofinstructions, algorithms, or software for controlling the operation andfunctions of decoder 230.

Memory 233 has stored therein video data 214, which has stored thereinSpanish subtitle data 208 and burned-in subtitle data 216, French audiodata 204, and English audio data 206. Memory 233 additionally hasinstructions stored therein to allow controller 231 to receive data fromencoder 200 and decode the encoded data.

Decoder 230 receives transport stream 218 from encoder 200. Controller231 preferably executes instructions stored on memory 233 to decodetransport stream 218. Decoder 230 may then transmit video data 214,which has stored therein Spanish subtitle data 208 and burn-in subtitledata 216, French audio data 204, and English audio data 206 to displaydevice 100. In some cases, decoder 230 and display device 100 may be aunitary device, as shown by dotted box 240.

As shown in FIG. 2 , decoder 230 transmits to display device 100 videodata 214, which has stored therein burned-in subtitle data 216, Frenchaudio data 204, English audio data 206, and Spanish subtitle data 208.Display device 100 may display video data 214 as a video. The video willcontain the contents of video data 202 in addition to burned-in subtitledata 216. If a user were to turn on the alternate subtitles, in the formof Spanish subtitle data 208, it is likely that they would appear in thesame place as burned-in subtitle data 216. As burned-in subtitle data216 is burned into the video, it cannot be removed as Spanish subtitledata 208 can. If the user has no use for burn-in subtitle data 216, thenit will merely obstruct their view of Spanish subtitle data 208. Thiscan be frustrating for users who want to select their own language forsubtitles. What is needed, therefore, is a system and method fordisplaying alternate subtitles in a location different from burned insubtitles.

Systems and methods in accordance with the present disclosure displayalternate subtitles in a location different from burned in subtitles.Specifically, in some disclosed embodiments, an encoder receives videodata, audio data, and subtitle data from an external source. Thesubtitle data may be burned into the video data, modifying it. Metadatamay be generated to determine the location of the burned in subtitledata. The encoder may receive supplemental audio data and subtitle datadifferent from the original audio and subtitle data. The encoder mayencode all the data and transmit it to a decoder. The decoder may decodethe data and superimpose the second set of subtitle data onto themodified video data. Because the decoder receives the metadataindicating the location of the burned-in subtitles, if other subtitlesare chosen to be displayed, the decoder may decode the other subtitlessuch that they will be displayed in a different location than the firstset of subtitles—which are burned into the video. The decoder maytransmit the data to a display device to be viewed by a user, whereinthe burned-in subtitles do not obstruct the view of the alternate chosensubtitles.

Exemplary systems and methods for displaying alternate subtitles in alocation different from burned in subtitles in accordance with aspectsof the present disclosure will now be described in greater detail withreference to FIG. 3 to FIG. 6C. Specifically, FIG. 3 shows an exemplarymethod 300 to be executed by a processor for encoding and transportingdata to a decoder in accordance with aspects of the present disclosure.As shown in this figure, method 300 begins (S302) and an encoderreceives video data (S304) as will be described in greater detail withreference to FIG. 4 , which illustrates an exploded view of encoder 400,transport stream 406, decoder 410, and display device 420 in accordancewith aspects of the present disclosure. In some embodiments, encoder 400may receive video data 202 from a data provider, a non-limiting exampleof which may be a cable modem termination system (CMTS).

As shown in FIG. 4 , encoder 400 includes a controller 401 and a memory402. Encoder 400 receives video data 202, French audio data 204, Englishaudio data 206, Spanish subtitle data 208, English subtitle data 210,and subtitle metadata 403. Memory 402 has stored therein video data 214and burn-in subtitle data 216. In some embodiments, at least one ofcontroller 401 and memory 402 may be implemented as a computer havingtangible computer-readable media for carrying or havingcomputer-executable instructions or data structures stored thereon. Suchnon-transitory computer-readable recording medium refers to any computerprogram product, apparatus or device, such as a magnetic disk, opticaldisk, solid-state storage device, memory, programmable logic devices(PLDs), DRAM, RAM, ROM, EEPROM, CD-ROM or other optical disk storage,magnetic disk storage or other magnetic storage devices, or any othermedium that can be used to carry or store desired computer-readableprogram code in the form of instructions or data structures and that canbe accessed by a general-purpose or special-purpose computer, or ageneral-purpose or special-purpose processor. Disk or disc, as usedherein, includes compact disc (CD), laser disc, optical disc, DVD,floppy disk and Blu-ray disc. Combinations of the above are alsoincluded within the scope of computer-readable media. For informationtransferred or provided over a network or another communicationsconnection (either hardwired, wireless, or a combination of hardwired orwireless) to a computer, the computer may properly view the connectionas a computer-readable medium. Thus, any such connection may be properlytermed a computer-readable medium. Combinations of the above should alsobe included within the scope of computer-readable media.

Example tangible computer-readable media may be coupled to a processorsuch that the processor may read information from and write informationto the tangible computer-readable media. In the alternative, thetangible computer-readable media may be integral to the processor. Theprocessor and the tangible computer-readable media may reside in anintegrated circuit (IC), an application specific integrated circuit(ASIC), or large-scale integrated circuit (LSI), system LSI, super LSI,or ultra LSI components that perform a part or all of the functionsdescribed herein. In the alternative, the processor and the tangiblecomputer-readable media may reside as discrete components.

Example tangible computer-readable media may also be coupled to systems,non-limiting examples of which include a computer system/server, whichis operational with numerous other general purpose or special purposecomputing system environments or configurations. Examples of well-knowncomputing systems, environments, and/or configurations that may besuitable for use with computer system/server include, but are notlimited to, personal computer systems, server computer systems, thinclients, thick clients, handheld or laptop devices, multiprocessorsystems, microprocessor-based systems, set-top boxes, programmableconsumer electronics, network PCs, minicomputer systems, mainframecomputer systems, and distributed cloud computing environments thatinclude any of the above systems or devices, and the like.

Such a computer system/server may be described in the general context ofcomputer system-executable instructions, such as program modules, beingexecuted by a computer system. Generally, program modules may includeroutines, programs, objects, components, logic, data structures, and soon that perform particular tasks or implement particular abstract datatypes. Further, such a computer system/server may be practiced indistributed cloud computing environments where tasks are performed byremote processing devices that are linked through a communicationsnetwork. In a distributed cloud computing environment, program modulesmay be located in both local and remote computer system storage mediaincluding memory storage devices.

Components of an example computer system/server may include, but are notlimited to, one or more processors or processing units, a system memory,and a bus that couples various system components including the systemmemory to the processor.

The bus represents one or more of any of several types of busstructures, including a memory bus or memory controller, a peripheralbus, an accelerated graphics port, and a processor or local bus usingany of a variety of bus architectures. By way of example, and notlimitation, such architectures include Industry Standard Architecture(ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA)bus, Video Electronics Standards Association (VESA) local bus, andPeripheral Component Interconnects (PCI) bus.

A program/utility, having a set (at least one) of program modules, maybe stored in the memory by way of example, and not limitation, as wellas an operating system, one or more application programs, other programmodules, and program data. Each of the operating system, one or moreapplication programs, other program modules, and program data or somecombination thereof, may include an implementation of a networkingenvironment. The program modules generally carry out the functionsand/or methodologies of various embodiments of the application asdescribed herein.

Controller 401 may be implemented as a hardware processor such as amicroprocessor, a multi-core processor, a single core processor, a fieldprogrammable gate array (FPGA), a microcontroller, an applicationspecific integrated circuit (ASIC), a digital signal processor (DSP), orother similar processing device capable of executing any type ofinstructions, algorithms, or software for controlling the operation andfunctions of encoder 400 in accordance with the embodiments described inthe present disclosure.

Memory 402, as will be described in greater detail below, hasinstructions stored therein to cause encoder 400 to generate video data214 based on an incorporation of the burn-in subtitle data 216 intovideo data 202 such that English subtitle data 210 will be displayed ata predetermined location when the modified video is displayed on displaydevice 420, as well as to generate subtitle metadata 403 associated withthe predetermined location, generate transport stream 406 includingencoded video data 220, encoded subtitle metadata 408 and encodedsubtitle data 228, and to transmit transport stream 406 to displaydevice 420 so as to enable display device 420 to decode transport stream406 and display video data 214 and Spanish subtitle data 208 such thatthe Spanish subtitle data 208 is not displayed at the predeterminedlocation.

In some embodiments, memory 402, as will be described in greater detailbelow, has instructions stored therein to cause encoder 400 to transmittransport stream 406 to display device 420 so as to enable displaydevice 420 to decode transport stream 406 and display video data 214 andencoded subtitle data 228 such that Spanish subtitle data 208 isdisplayed at a second predetermined location, wherein the predeterminedlocation is different from the second predetermined location.

In some embodiments, memory 402, as will be described in greater detailbelow, has instructions stored therein to cause encoder 400 to transmittransport stream 406 to display device 420 so as to enable displaydevice 420 to decode transport stream 406 and display video data 214 andSpanish subtitle data 208 such that Spanish subtitle data 208 isdisplayed at a second location, wherein the predetermined location isdifferent from the second location.

In some embodiments, memory 402, as will be described in greater detailbelow, has instructions stored therein to cause encoder 400 to generatetransport stream 406 including French audio data 204 and English audiodata 206.

Subtitle metadata 403 contains the location of burned-in subtitle data216. Subtitle metadata 403 preferably ensures that Spanish subtitle data208 is displayed in a different location than that of burned-in subtitledata 216. In some embodiments, subtitle metadata 403 is provided toencoder 400 from an external source, such as the content provider thatis providing video data 202, French audio data 204, English audio data206, Spanish subtitle data 208, and English subtitle data 210. However,in some embodiments, controller 401 may generate subtitle metadata 403when burned-in subtitle data 216 is created.

Transport stream 406 includes encoded video data 220, which has storedtherein encoded burned-in subtitle data 222, encoded audio data 224,encoded audio data 226, encoded subtitle data 228, and subtitle metadata403. Transport stream 406 contains the encoded data sent from encoder400 to decoder 410. While the encoded data is labeled with new numbers,the data is the same. Once decoded, the data will revert back to itsoriginal state.

Decoder 410 includes: a controller 411 and a memory 412. Memory 412contains video data 214, which has stored therein burn-in subtitle data216, French audio data 204, English audio data 206, Spanish subtitledata 208, and subtitle metadata 403. In some embodiments, at least oneof controller 411 and memory 412 may be implemented as a computer havingtangible computer-readable media for carrying or havingcomputer-executable instructions or data structures stored thereon.

Controller 411 may be implemented as a hardware processor such as amicroprocessor, a multi-core processor, a single core processor, a fieldprogrammable gate array (FPGA), a microcontroller, an applicationspecific integrated circuit (ASIC), a digital signal processor (DSP), orother similar processing device capable of executing any type ofinstructions, algorithms, or software for controlling the operation andfunctions of decoder 410 in accordance with the embodiments described inthe present disclosure.

Memory 412, as will be described in greater detail below, hasinstructions stored therein to cause decoder 410 to decode transportstream 406 to generate video data 214, subtitle metadata 403, andSpanish subtitle data 208, and also to provide video data 214 andSpanish subtitle data 208 to display device 420 such that, based onsubtitle metadata 403, Spanish subtitle data 208 is not displayed at thepredetermined location.

In some embodiments, memory 412, as will be described in greater detailbelow, has instructions stored therein to cause decoder 410 to transmittransport stream 406 to display device 420 so as to enable displaydevice 420 to provide video data 214 and Spanish subtitle data 208 todisplay device 420 such that, based on subtitle metadata 403, Spanishsubtitle data 208 is displayed at a second predetermined location, andwherein the predetermined location is different from the secondpredetermined location.

Decoder 410 may receive transport stream 406 from encoder 400.Controller 411 may execute instructions stored on memory 412 to causedecoder 410 to decode transport stream 406. Decoder 410 may thentransmit video data 214, which has stored therein Spanish subtitle data208 and burn-in subtitle data 216, French audio data 204, English audiodata 206, and subtitle metadata 403 to display device 420.

Display device 420 may be any known device or system to display an imageto a user. In some embodiments, decoder 410 and display device 420 willbe a unitary device, as shown by dotted box 430.

Returning to FIG. 3 , after the encoder receives the video data (S304),the encoder preferably receives the supplemental data (S306). Forexample, presume that encoder 400 also receives French audio data 204,English audio data 206, Spanish subtitle data 208, and English subtitledata 210 from a CMTS. French audio data 204 and English audio data 206are associated with Spanish subtitle data 208 and English subtitle data210 respectively, as referenced in FIG. 4 .

After the encoder receives the supplemental data (S306), modified videodata is generated (S308). In an example embodiment, the modified videodata corresponds to a modified video which includes the original videohaving subtitles burned in. For example, with reference to FIG. 4 ,controller 401 may execute instructions stored on memory 402 to allowencoder 400 to alter video data 202. English subtitle data 210 maytherefore be burned into video data 202, creating video data 214, whichis stored in memory 402 as burned-in subtitle data 216.

Returning to FIG. 3 , after the modified video data is generated (S308),the metadata is generated (S310). For example, with reference to FIG. 4, encoder 400 will also generate subtitle metadata 403. After subtitledata 216 is burned into video data 214, controller 401 may executeinstructions stored on memory 402 to allow encoder 400 to determine thepredetermined location of burned-in subtitle data 216. Thispredetermined location is the location for which the burned-in subtitleswill be displayed. This will be described in greater detail withreference to FIG. 6A, which illustrates a display device 420 at a timet₀.

As shown in FIG. 6A, display device 420 may display an image 602,predetermined subtitle location 608, English subtitles 610, subtitlelocation 614, and subtitles 616. Predetermined subtitle location 608corresponds to the predetermined location of where the subtitles will bedisplayed. As such, subtitle metadata 403 is the location, on the videodisplay, where the subtitles corresponding to burned-in subtitle data216 will be displayed. As shown in the figure, the location where thesubtitles corresponding to burned-in subtitle data 216 is predeterminedsubtitle location 608.

In accordance with aspects of the present disclosure, in someembodiments, controller 401 executes instruction in memory 402 togenerate subtitle metadata 403 when English subtitle data 210 is beingburned into video data 202 to create video data 214 with burned insubtitle data 216. As noted above, in some embodiments, controller 401generates subtitle metadata 403 by receiving subtitle metadata 403 froman external source. In such embodiments, controller 401 then executesinstruction in memory 402 to burn subtitle data 210 into video data 202in accordance with the received metadata 403.

Returning to FIG. 3 , after the metadata is generated (S310), it may bedetermined if there is second supplemental data available (S312). Forexample, controller 401 may execute instructions stored on memory 402 toallow encoder 400 to determine if there is subtitle data in addition toEnglish subtitle data 210. In some cases, a media provider may providemultiple subtitles to reach markets of various languages. In thisnon-limiting example, Spanish subtitle data 208 is additionallyprovided.

If it is determined there is no second supplemental data available (N atS312), then an encoded transport stream is generated (S316). Forexample, with reference to FIG. 4 , presume that encoder 400 onlyreceives English subtitle data 210, controller 401 may executeinstructions stored on memory 402 to allow encoder 400 to encode videodata 214, which has stored therein burned-in subtitle data 216, Englishaudio data 206, and subtitle metadata 403. The encoded data is shown astransport stream 406.

It should be noted that encoder 400 may encode data to create transportstream 406 by any known encoding scheme, a non-limiting example of whichincludes MPEG. It should also be noted that encoder 400 may encode datato create transport stream 406 in a manner in which decoder 410 isconfigured to decode the encoded data as transport stream 406.

Returning to FIG. 3 , after an encoded transport stream is generated(S316), the encoded transport stream is transmitted (S318). For example,with reference to FIG. 4 , encoder 400 may transmit transport stream 406to decoder 410 over communication channel 217. Video data 214, burned-insubtitle data 216, English audio data 206 and subtitle metadata 403 areencoded into encoded video data 220, encoded burned-in subtitle data222, encoded audio data 226 and encoded subtitle metadata 408respectively.

Returning to FIG. 3 , after the encoded transport stream is transmitted(S318), method 300 stops (S320). In this case, the transmitted encodeddata does not include any subtitle data other than the subtitle datathat is burned into the video data. As such, there is no threat of auser choosing to use another set of subtitles that will be displayedover the burned-in subtitles, as discussed above with reference to FIG.1C.

If it is determined there is second supplemental data (at S312), it isreceived (S314). For example, with reference to FIG. 4 , presume thatencoder 400 additionally receives Spanish subtitle data 208; withsubtitle metadata 403, display device 420 will be able to displaySpanish subtitle data 208 in a location different from location 106 ofsubtitles 108 as shown in FIG. 1B corresponding to burned-in subtitledata 216.

Returning to FIG. 3 , after the second supplemental data is received(S314), then an encoded transport stream is generated (S316) and theencoded transport stream is transmitted (S318). Both of these processesmay be performed in a manner similar to that discussed above.

Once transmitted, a display device may receive the encoded video dataand display the video data. More importantly, in accordance with aspectsof the present disclosure, the displayed video may include the burned-insubtitles but may also include other subtitles that are not superimposedon the burned-in subtitles. As such, in accordance with aspects of thepresent invention, the displayed video may include subsequently chosensubtitles that are easily viewably because they are not superimposed onat the same location within the video as are those of the burned-insubtitles. This will be described in greater detail with reference toFIGS. 5-6C.

FIG. 5 illustrates a method 500 to be executed by a processor forreceiving encoded data from an encoder and decoding data in accordancewith aspects of the present disclosure. Specifically, the method starts(S502) and the encoded transport stream is received (S504). For example,with reference to FIG. 4 , decoder 410 receives transport stream 406from encoder 400.

Returning to FIG. 5 , after the encoded transport stream is received(S504), the encoded transport stream may be decoded (S506). For example,with reference to FIG. 4 , controller 411 may execute instructionsstored on memory 412 to allow decoder 410 to decode transport stream406. It should be noted that decoder 410 may decode the encoded data byany known decoding scheme, a non-limiting example of which includesMPEG. It should also be noted that decoder 410 may decode the encodeddata in transport stream 406 in a manner corresponding to which encoder400 is configured to encode the original data as transport stream 406.

Returning to FIG. 5 , after the encoded transport stream is decoded(S506), it may be determined if there is subtitle metadata (S508). Forexample, with reference to FIG. 4 , controller 411 may executeinstructions stored on memory 412 to allow decoder 410 to determine ifthere is subtitle metadata available. In some embodiments, encoder 400may encode data to create transport stream 406 to include a metadatafield, which may include a header and a payload. In some of theseembodiments, the header may be a predetermined number of bits. In someof these embodiments, the header may indicate that metadata is presentin transport stream 406. In some embodiments, the header may indicatethe size of the payload within transport stream 406. In someembodiments, the header may indicate the location of the payload withintransport stream 406. The payload may indicate the location for whichthe supplemental data is located, or in other words, the location towhich the burned-in subtitles will be displayed on the video when thevideo is played.

Returning to FIG. 5 , if it is determined that there is no subtitlemetadata available (N at S508), the video data may be provided to thedisplay device (S510). For example, with reference to FIG. 4 , presumethat subtitle metadata 403 was not included in transport stream 406 fromencoder 400. Decoder may then transmit video data 214, burned-insubtitle data 216, and English audio data 206 to display device 420.

Returning to FIG. 5 , after the video data is provided to the displaydevice (S510), method 500 stops (S518). In this situation, the videowill merely be played with the original burned-in subtitles displayed ina manner similar to that discussed above with reference to FIG. 1B.

Returning to FIG. 5 , if it is determined that there is subtitlemetadata (Y at S508), then the predetermined location of thesupplemental data is preferably determined (S512). For example, withreference to FIG. 4 , presume that decoder 410 determines that subtitlemetadata 403 is included in transport stream 406. Controller 411 mayexecute instructions stored on memory 412 to allow decoder 410 toanalyze subtitle metadata 403, access the payload of the metadata, anddetermine the location of the video at which the subtitles,corresponding to burned-in subtitle data 216, will be displayed on thevideo when the video is played.

Returning to FIG. 5 , after the predetermined location of thesupplemental data is determined (S512), it is preferably determined ifthere is second supplemental data requested (S514). For example, decoder410 may determine if the user watching display device 420 has requestedfor the subtitles associated with Spanish subtitle data 208. The usermay request additional subtitles by any known method, a non-limitingexample of which includes a commonly used user interface menu (notshown) that may be accessed by the user to choose additional subtitles.

Returning to FIG. 5 , if it is determined that there is no secondsupplemental data requested (N at S514), then the video data is providedto the display device (S510). For example, with reference to FIG. 4 ,decoder 410 may determine that the user watching display device 420 hasnot requested the subtitles associated with Spanish subtitle data 208.In this situation, although Spanish subtitles are offered, the user doesnot want to use them when watching the video. As such, only burned-insubtitle data 216 will be displayed, decoder 410 will then transmitvideo data 214, which has stored therein burn-in subtitle data 216, andEnglish audio data 206 to display device 420. Again, in this situation,the video will merely be played with the original burned-in subtitlesdisplayed in a manner similar to that discussed above with reference toFIG. 1B.

Returning to FIG. 5 , if it is determined that there is secondsupplemental data requested (Y at S514), then the second supplementaldata is superimposed onto the modified video data (S516). For example,if decoder 410 determines the user watching display device 420 hasrequested the subtitles associated with Spanish subtitle data 208 andFrench audio data 204, controller 411 may execute instructions stored onmemory 412 to instruct decoder 410 to superimpose the Spanish subtitles,corresponding to Spanish subtitle data 208, at a location of the videothat is different from the location of the burned-in subtitles,corresponding to subtitle data 216.

In some embodiments, subtitle metadata 403 includes a “HERE”instruction. The HERE instruction includes location instruction data toinstruct decoder 410 as to the specific location where the Spanishsubtitles, corresponding to Spanish subtitle data 208, should besuperimposed on the video. The specific location will be located on thevideo such that the Spanish subtitles are not superimposed on the videoat the location of the burned-in subtitles, corresponding to subtitledata 216

In some embodiments, subtitle metadata 403 includes an “ANYWHERE BUTHERE” or NOT HERE instruction. The ANYWHERE BUT HERE instructionincludes location instruction data to instruct decoder 410 to notsuperimpose the Spanish subtitles, corresponding to Spanish subtitledata 208, so as to be superimposed on the video at the location of theburned-in subtitles, corresponding to subtitle data 216. The ANYWHEREBUT HERE instruction embodiment differs from the HERE instructionembodiment discussed above in that in the ANYWHERE BUT HERE instructionembodiment, decoder 410 determines where to superimpose the Spanishsubtitles. This determination may be implemented by any known process, anon-limiting example of which includes an a priori determination. On theother hand, in the HERE instruction embodiment, encoder 400 determineswhere to superimpose the Spanish subtitles. This determination may beimplemented by any known process, a non-limiting example of whichincludes an a priori determination.

Returning to FIG. 5 , after the second supplemental data is superimposedonto the modified video data (S516), the video data is provided to thedisplay device (S510). For example, decoder 410 will transmit video data214, burn-in subtitle data 216, French audio data 204, English audiodata 206, and Spanish subtitle data 208 to display device 420. If a userturns on subtitles associated with Spanish subtitle data 208, they willthen appear in a different location from the burned in subtitlesassociated with burn-in subtitle data 216. This will be described ingreater detail with reference to FIGS. 6A-C.

As mentioned above, FIG. 6A illustrates display device 420 at a time t₀.As shown in FIG. 6A, predetermined subtitle location 608 is separatedfrom subtitle location 614. English subtitles 610, associated withburn-in subtitle data 216, are displayed above Spanish subtitles 616,associated with subtitle data 208. For example, a user may watch videoassociated with video data 214. If the user chooses Spanish subtitles,then Spanish subtitles 616, associated with subtitle data 208, will besuperimposed on the video so as to be displayed to the user in alocation different from the location of English subtitles 610. In thefigure, Spanish subtitles 616 are displayed directly below Englishsubtitles 610, but Spanish subtitles 616 can be superimposed at anylocation that does not conflict with subtitle location 614.

In the example embodiments discussed above, non-limiting examples ofEnglish, French, and Spanish languages have been described. It should benoted that these examples are non-limiting and that any number oflanguages may be used in accordance with aspects of the presentdisclosure. Furthermore, as indicated previously, though displayed textin the form of subtitles has been used as an example, the disclosedsystems and methods may be used with respect to other text that may beeither burned-in or selectively displayed at the option of a user, suchas additional contextual information about a displayed image, scrollingstock tickers, emergency alerts, etc. Thus, any image data may be usedas the supplemental data in accordance with aspects of the presentdisclosure. Similarly, the second supplemental data that is associatedwith metadata and that corresponds to an image that will be superimposedon the modified video may be any image data a in accordance with aspectsof the present disclosure. This will be described in greater detail withreference to FIGS. 6B-C.

FIG. 6B illustrates another example embodiment of display device 420 ata time t₁ where display device 420 displays an image 602, predeterminedsubtitle location 608, English subtitles 610, supplemental data location618, and a supplemental image 620. In this example embodiment, thesecond supplemental data associated with the metadata does notcorrespond to subtitles. Instead, in this embodiment, the secondsupplemental data associated with the meta data corresponds to networkemblem or logo, shown as supplemental image 620. As shown in FIG. 6B,supplemental data location 618 superimposes supplemental image 620 at adifferent location from English subtitles 610, associated with burned-insubtitle data 216.

For example, presume that a network wants to include their logo within amovie that is provided to the viewer. The content provided by thenetwork will include second supplemental data corresponding tosupplemental image 620, such that supplemental image 620 be displayed toa user watching the video in a location different from the location ofEnglish subtitles 610.

FIG. 6C illustrates another example embodiment of display device 420 ata time t₂ where display device 420 displays an image 602, predeterminedsubtitle location 608, English subtitles 610, and supplemental datalocation 622, and information 624. In this example embodiment, thesecond supplemental data associated with the metadata, again, does notcorrespond to subtitles. Instead, in this embodiment, the secondsupplemental data associated with the metadata corresponds to scrollinginformation, shown as information 624. As shown in FIG. 6C, supplementaldata location 622 superimposes information 624 at a different locationfrom English subtitles 610, associated with burned-in subtitle data 216.

For example, presume that a network wants to include a tickertape ofsports scores, stock prices, major news events, etc., within a moviethat is provided to the viewer. The content provided by the network willinclude second supplemental data corresponding to information 624, suchthat information 624 be displayed to a user watching the video in alocation different from the location of English subtitles 610.

Some visual media contains subtitles that are burned into video, whereinthey cannot be removed. If a user wants to use a different set ofsubtitles, it may prove to be difficult as the subtitles may overlap.This will obstruct the viewer's view of the subtitles that they arelooking to use. In accordance with the present disclosure, systems andmethods prevent non-burned-in subtitles from overlapping burned-insubtitles when they are presented at the same time on a video. Thisapproach embeds and transmits, in the encoder side, spatial and temporalinformation related to the burnt-in subtitles as metadata. Thisinformation may be used a priori, on the decoder side to avoid textoverlapping when displaying other non-burnt-int subtitles on the video.

In an example embodiment, an encoder may receive video data, and twosets of audio data and respective subtitle data. The first set ofsubtitle data will be burned into the video data, altering it. Theencoder will generate subtitle metadata, detailing the location of theburned in subtitle data. The encoder will encode all the data andtransmit it to a decoder. The decoder will decode the data, superimposethe alternate subtitles onto the modified video data. All of the datawill then be transmitted to a display device. If a user turns on thealternate subtitles, they will display in a different location than thatof the burned in subtitles. Thus, the present disclosure as discloseddisplays alternate subtitles in a different location than burned insubtitles.

Also, as previously indicated, content or supplemental data such asscrolling text (text crawls), logo's or generally speaking any contentsupplemental to a video may be burned in at a specific location in oneor more sequential frames of video, and when this occurs, that burned-incontent may interfere with the legibility of both that content as wellas that of any overlapping subtitles that are chosen to be displayed bya viewer. This condition may occur when the chosen subtitles overlapwith the region in which the burned-in content is displayed. As caneasily be appreciated, the foregoing techniques may be used to encodemetadata into a transport stream of the video, specifying the locationof the burned-in content, so as to enable a decoder to relocate anyselected subtitles to a location that does not overlap the burned-incontent.

FIG. 7 for example, shows a system 700 having an encoder 702 thatencodes a transport stream 704 delivered to a decoder 706. The encoder702 receives as separate inputs a video feed 708, one or more audioinputs 710 (Audio 1, Audio 2, . . . Audio n) and one or more subtitleinputs 712 (Subtitles 1, Subtitles 2 . . . Subtitles n). Typically, theaudio inputs 710 will comprise soundtracks having one or more spokenlanguages (English DTS, English Dolby, French DTS, Spanish Dolby, etc.),while the subtitle tracks 712 will similarly comprise text correspondingto different languages to allow a viewer to read text of what is spokenin the video, or otherwise explain what is occurring in the video when,say, the viewer is either hearing-impaired or does not understand thelanguage of the audio track selected for the presentation.

The encoder 702 may also selectively receive a text crawl input 707. Atext crawl is a text that passes over or into the screen, which is oftenused to transmit special announcements while having minimal impact onthe main video content. For example, the text crawl input may correspondto an emergency alert, stock or sports tickers, weather information,notifications of upcoming programming, or generally speaking anysupplemental information for which a viewer is to be informed.Typically, the texts crawl scrolls left to right across the either topor the bottom of a display and is usually called e.g., the crawl,crawler, slide, or news ticker.

In the system 700, the encoder 702 burns the text crawl 707 into thevideo, meaning that the frames of video 716 output to the transportstream 704 have pixel data that includes the text crawl 707 so that whenthe decoder 706 reconstructs the video frames, the text crawl will beunavoidably displayed on screen. Therefore, consonant with the foregoingdisclosure related to FIGS. 3-5 , the encoder 702 will also generatemetadata 714 indicating both the presence and location of the text crawlin the video frames. Upon receipt of the transport stream, the decoder706 is therefore able to retrieve 722 the metadata 714, and when a userselects 720 a subtitle track (in this instance “Subtitles 2”) thedecoder is able to position 724 the selected subtitles to a locationthat does not overlap with the burned-in text crawl.

As also indicated previously, the metadata 714 may indicate the positionof the burned-in content, such as the text crawl 714 by indicating wherethe burned-in subject matter is not (e.g., a HERE instruction) oralternatively indicate where the burned-in subject matter is located(e.g., an ANYWHERE BUT HERE instruction). In some embodiments, given thenature of a text crawl being a horizontally-scrolling, narrow ticker,these instructions may be implemented by a binary parameter or value(e.g., top or bottom of the screen).

FIG. 8 shows an alternate system 800 having an encoder 802 that encodesa transport stream 804 delivered to a decoder 806. The encoder 802receives as separate inputs a video feed 808, one or more audio inputs810 (Audio 1, Audio 2, . . . Audio n) and one or more subtitle inputs812 (Subtitles 1, Subtitles 2 . . . Subtitles n). Typically, the audioinputs 810 will comprise soundtracks having one or more spoken languages(English DTS, English Dolby, French DTS, Spanish Dolby, etc.), while thesubtitle tracks 812 will similarly comprise text corresponding todifferent languages to allow a viewer to read text of what is spoken inthe video, or otherwise explain what is occurring in the video when,say, the viewer is either hearing-impaired or does not understand thelanguage of the audio track selected for the presentation.

The encoder 802 may also selectively receive a logo or other suchgraphical overlay input 807 that supplements the presentation displayed.For example, the logo may be a network logo, a content notification suchas TV-14, etc. that are usually transitorily displayed on screen, butsometimes in the case of network logos, permanently displayed.Typically, these logos or graphical displays are located in one of thefour corners of the frames of video. For purposes of discussing thisembodiment, reference to the supplementary content or graphical overlaywill be referred to as a “logo,” but as just indicated, other types ofoverlays are envisioned with this embodiment.

In the system 800, the encoder 802 burns the logo 807 into the video,meaning that the frames of video 716 output to the transport stream 804have pixel data that includes the logo 807 so that when the decoder 806reconstructs the video frames, the logo will be unavoidably displayed onscreen. Therefore, consonant with the foregoing disclosure related toFIGS. 3-5 , the encoder 802 will also generate metadata 814 indicatingboth the presence and location of the logo in the video frames. Uponreceipt of the transport stream, the decoder 806 is therefore able toretrieve 822 the metadata 814, and when a user selects 820 a subtitletrack (in this instance “Subtitles 1”) the decoder is able to position824 the selected subtitles to a location that does not overlap with theburned-in logo.

As also indicated previously, the metadata 814 may indicate the positionof the burned-in content, such as the logo 814 by indicating where theburned-in subject matter is not (e.g., a HERE instruction) oralternatively indicate where the burned-in subject matter is located(e.g., an ANYWHERE BUT HERE instruction). In some embodiments, given thenature of a logo being positioned in one of the corners of the frame,these instructions may be implemented by a discrete parameter (e.g., topleft, top right, bottom left, or bottom right of the screen).

Those of ordinary skill in the art will appreciate that, although theclaimed burned-in supplemental material was described using severaldifferent distinct exemplary embodiments i.e., burned-in subtitles,burned-in text crawls, and burned-in logos, other embodiments may usedifferent types of burned-in material where content supplemental toreceived video data is encoded into the pixel data of a video. Moreover,although these different examples were discussed individually, otherembodiments may employ a combination of different types of burned-incontent. For example, both burned-in subtitles and burned-in logos maybe present, etc. In this instance, the disclosed metadata encoded andsent to the decoder may indicate the existence and position of eachtype, which in some embodiments may comprise a HERE instructionspecifying the location of user-selected subtitles, or alternately a NOTHERE instruction specifying the location(s) of burned-in content.

Moreover, the metadata 403, 714, 814 may in some embodiments beassociated with an entire presentation, e.g., a movie or in otherembodiments may be associated with specific frames, groups of pictures,segments of adaptive bitrate content sent as transport stream(.ts)files, etc. In this manner, for example, as logos may potentially appearand disappear on the screen, or as emergency or upcoming contentnotification text scrolls are periodically broadcast, the location ofuser-selected text may periodically move to whatever location may berequired at that time so as to not overlap with burned-in content.

Still further, although the embodiments previously disclosed in thisspecification described encoders that received supplemental content,such as subtitles, crawl text, logos, etc. that was to be burned in bythe encoder, other embodiments may employ encoders or transcoders thatreceive frames of video with such supplemental content already burnedin, but lacking the necessary metadata by which a decoder will benotified of the burned-in content and/or will be able to repositionuser-selected subtitles. In this circumstance the encoder may beconfigured to detect the burned-in content, and its location, and createthe necessary metadata.

For example, the encoder may be configured to buffer a plurality offrames and analyze them for any static content across the plurality offrames that, for example, does not match the global motion of a framewhile maintaining the same absolute position in the frame. Other suchembodiments may analyze frames that span across a scene cut to determinestatic content that survives across the scene cut. In still otherembodiments, text recognition algorithms may be used to detect theexistence of subtitles, and/or templates may be used to detect scrollingtext and/or logos. These examples are merely illustrative of the manytechniques that may be used to detect supplementary content that hasbeen burned into a sequence of video frames. Once the burned-insupplementary content has been detected, the encoder may create metadatathat indicates its existence and position as described earlier.

Still other embodiments may comprise a decoder that is configured toitself detect burned in content without first receiving metadata. Suchan encoder may similarly be configured to buffer a plurality of framesand analyze them for any static content across the plurality of framesthat, for example, does not match the global motion of a frame whilemaintaining the same absolute position in the frame. Other suchembodiments may analyze frames that span across a scene cut to determinestatic content that survives across the scene cut. In still otherembodiments, text recognition algorithms may be used to detect theexistence of subtitles, and/or templates may be used to detect scrollingtext and/or logos. These examples are merely illustrative of the manytechniques that may be used to detect supplementary content that hasbeen burned into a sequence of video frames. Once any burned-in materialhas been detected, the decoder according to this embodiment maydetermine its position, using any metric (e.g., pixel coordinateboundary, discrete locations such as top, bottom, left corner, etc.),and thereafter reposition any user-selected subtitles so that they donot overlap with the detected burned-in material.

The operations disclosed herein may constitute algorithms that can beaffected by software, applications (apps, or mobile apps), or computerprograms. The software, applications, computer programs can be stored ona non-transitory computer-readable medium for causing a computer, suchas the one or more processors, to execute the operations describedherein and shown in the drawing figures.

The foregoing description of various preferred embodiments have beenpresented for purposes of illustration and description. It is notintended to be exhaustive or to limit the invention to the precise formsdisclosed, and obviously many modifications and variations are possiblein light of the above teaching. The example embodiments, as describedabove, were chosen and described in order to best explain the principlesof the invention and its practical application to thereby enable othersskilled in the art to best utilize the invention in various embodimentsand with various modifications as are suited to the particular usecontemplated. It is intended that the scope of the invention be definedby the claims appended hereto.

What is claimed is:
 1. An encoder comprising: an input configured toreceive a plurality of frames of video, at least one subtitle track, andsupplemental data that relates to the content shown in the plurality offrames; and a processor configured to encode the frames of video withthe supplemental content burned-in, and to create metadata indicating atleast one of the existence of and the position of the burned-insupplemental content; and an output configured to output the encodedframes video with the supplemental content burned-in, the at least onesubtitle track, and the metadata.
 2. The encoder of claim 1 where thesupplemental content comprises subtitles.
 3. The encoder of claim 1where the supplemental content comprises a text crawl.
 4. The encoder ofclaim 1 where the supplemental content comprises a logo.
 5. The encoderof claim 1 where the metadata comprises a selected one of an instructionto a decoder to place a user-selected one of the at least one subtitletrack at a defined location or an instruction to a decoder not to placea user-selected one of the at least one subtitle track at a definedlocation.
 6. The encoder of claim 1 where the metadata is associatedwith a subset of the plurality of frames of video.
 7. The encoder ofclaim 1 where the metadata is associated with all the plurality offrames of video.
 8. The encoder of claim 1 where the existence of, andposition of, the burned-in supplemental content is indicated by aselected one of a plurality of discrete, predetermined values.
 9. Theencoder of claim 1 where the existence of, and position of, theburned-in supplemental content is indicated by a boundary within aframe.
 10. The encoder of claim 1 where the metadata identifies aplurality of different positions for the burned-in supplementarycontent.
 11. A decoder comprising: a first input configured to receive aplurality of frames of video, at least one subtitle track, and metadataindicating at least one of the existence of and the position ofburned-in supplemental content in the plurality of frames of video; asecond input configured to receive a selection by a user of a selectedone of the at least one subtitle track; and a processor configured torespond to the selection of the user by using the metadata to repositionsubtitles associated with the selected subtitle track.
 12. The decoderof claim 11 where the supplemental content comprises subtitles.
 13. Thedecoder of claim 11 where the supplemental content comprises a textcrawl.
 14. The decoder of claim 11 where the supplemental contentcomprises a logo.
 15. The decoder of claim 1 where the metadatacomprises a selected one of an instruction to a decoder to place auser-selected one of the at least one subtitle track at a definedlocation or an instruction to a decoder not to place a user-selected oneof the at least one subtitle track at a defined location.
 16. Thedecoder of claim 11 where the metadata is associated with a subset ofthe plurality of frames of video.
 17. The decoder of claim 11 where themetadata is associated with all the plurality of frames of video. 18.The decoder of claim 11 where the existence of, and position of, theburned-in supplemental content is indicated by a selected one of aplurality of discrete, predetermined values.
 19. The decoder of claim 11where the existence of, and position of, the burned-in supplementalcontent is indicated by a boundary within a frame.
 20. The decoder ofclaim 11 where the metadata identifies a plurality of differentpositions for the burned-in supplementary content.